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<N Abstract 



The nematode Caenorhabditis elegans is a well-known model organism used to investigate fundamental 

questions in biology. Motility assays of this small roundworm are designed to study the relationships 

between genes and behavior. Commonly, motility analysis is used to classify nematode movements and 

1—^ characterize them quantitatively. Over the past years, C. elegans' motility has been studied across a wide 

^>- range of environments, including crawling on substrates, swimming in fluids, and locomoting through 

rj microfluidic substrates. However, each environment often requires customized image processing tools 

^ relying on heuristic parameter tuning. In the present study, we propose a novel Multi-Environment 

Q Model Estimation (MEME) framework for automated image segmentation that is versatile across various 

'""' environments. The MEME platform is constructed around the concept of Mixture of Gaussian (MOG) 

I models, where statistical models for both the background environment and the nematode appearance are 

^ explicitly learned and used to accurately segment a target nematode. Our method is designed to simplify 

OO the burden often imposed on users; here, only a single image which includes a nematode in its environment 

^»^ must be provided for model learning. In addition, our platform enables the extraction of nematode 

'skeletons' for straightforward motility quantification. We test our algorithm on various locomotive 

environments and compare performances with an intensity-based thresholding method. Overall, MEME 

outperforms the threshold-based approach for the overwhelming majority of cases examined. Ultimately, 

MEME provides researchers with an attractive platform for C. elegans' segmentation and 'skeletonizing' 

across a wide range of motility assays. 

> 

k> Introduction 

Cd Since its introduction in the laboratory over thirty years ago [5J, the nematode Caenorhabditis elegans 

has become a ubiquitous model organism to study fundamental questions in biology [39]. In particular. 



C. elegans is now widely used as a platform for drug screening and development 25l 31 , as well as for 



modeling various aspects of human diseases [8j|43] . In the quest to understand the relationships between 
genes and behavior, this small, approximately 1 mm long roundworm offers a number of advantages for 
laboratory applications. These include a short life cycle, the availability of many mutants to explore 



gene functions, knowledge of its complete cell lineage 45 46 , simplicity of the nervous system and its 



wiring |54|, and a fully sequenced genome 10 



A widespread strategy to investigate the genetic basis of behavior is to classify nematode movements 
and characterize them quantitatively. Traditionally, motility quantification has been based on crawling 



assays 27 35 36 51 , where C. elegans is observed to crawl on a substrate (e.g. agar plate). This is 
shown for example in Fig. flla). In the recent past, however, the number of environments used for nema- 
tode motility assays has vastly expanded. Studies of C. elegans' motility behavior now include various 
swimming assays [3|[T9{[28|[35||47[|48] , as shown in Fig.[T|b)-(d) and (g). In parallel, with the widespread 



availability of niicrofabrication techniques, nematode motility assays are increasingly conducted in mi- 
crofluidic environments [9p2p3l|37]. An example of such environments is shown in Fig.flVe) and (f). This 



latter platform has become particularly attractive for high-throughput drug screening applications [40|42 



Overall, with the growing variations in environments used for nematode behavioral assays, users are in 
need of reliable image analysis tools capable of extracting quantitative data across a wide spectrum of 
experimental mediums. 

The analysis of motility behavior has traditionally relied on qualitative observations to describe C 
elegans^ locomotion and discriminate between wild-type and mutant nematodes. In many instances, 
however, qualitative variations between strains are not apparent to the trained eye (as in pi). Such 
limitations have sprouted the development of automated image analysis systems in an effort to deliver 
relevant phenotypic differences between nematode strains [lj[6{[7{ [TH[T4|[l8|[20|j22|[53] . While the bulk of 
the research effort has been directed at analyzing locomotive traits of individual nematodes, some multi- 
worm tracking and feature extraction systems have also been developed l38Jl4ll[52] . Yet, the majority of 
state-of-the-art image analysis systems are designed for a specific environment, most commonly crawling 
[T pT|[l4p8|[2l|[2 2 , 38J or swimming assays [7 52 . These systems induce a tradeoff between either limiting 
the range of possible assays a researcher will investigate for motility analysis or customizing segmentation 
parameter selection across varying environments. However, an optimal system for the user is one which 
ideally bypasses such compromise. 

Current image analysis systems provide users with morphological and locomotion features to quantify 
behavioral phenotypes of C. elegans. 
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Such features include amongst other nematode speed 
wavelength and frequency of body undulations [7 11, body curvature [1 14 , and omega bends 
several of which make use of nematode center line data, also known as 'skeletons'. In practice, features 
are extracted from binary images, or segmentations, separating the nematode from its environment, 
or background. Several analysis systems compute binary images by applying a simple intensity-based 
threshold at each pixel location 
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Most commonly, this involves having the user 
manually select an appropriate range of intensities which characterizes the nematode. A variation to 
this approach has been the use of an adaptive threshold where nematode intensities, or appearance, are 
assumed to significantly differ from the average background intensities [I 21 22 . While these methods 
have shown promising capabilities, the range of environments for which they can be used for is in fact 
limited. This limitation is illustrated in Fig. [2] where the pixel intensity distributions of the nematode and 
background are respectively plotted for the environments shown in Fig.jT] Here, distributions are assumed 
to be Gaussian and parametrized with the mean and standard deviation of pixel intensities. In none of 
the cases shown can a single threshold separate any pair of distributions without causing significant 
errors (Fig. [2]). While threshold-based techniques can still be used to compute accurate segmentations, 
this requires significant effort on the user-end to adjust appropriate threshold values along with other 
noise canceling schemes {e.g. median filters, morphological operators, background subtraction, etc.). 
Altogether, this tedious process makes thresholding ill-suited for applications in complex background 
environments. 

More recently, alternative approaches to thresholding techniques have been pursued. For example, the 



work of Stauffer and Grimson 44 has been applied to the problem of nematode segmentation 41 , 52 



Here, the idea is to systematically learn how background pixels are individually distributed and use 
this information to segment the nematode. The learning process is done using a set of training images 
to statistically model the appearance of the background. Namely, each pixel is modeled by means of a 
Mixture of Gaussians (MOG), where the parameters of the model are learned from the training image set. 
This approach has been recently shown to provide excellent results for nematode segmentation |,41„52 as 

A major drawback, however, of modeling pixels with MOGs is 



well as for other applications 34 44 50 



that many parameters must be learned; this requires a large set of nematode-free training images. This 
condition largely prohibits extracting nematode segmentations from arbitrary sequences {e.g. open-access 
material) . 



In the present study, we propose a novel framework for image analysis of C elegans that is versatile 
across a wide range of environments. Moreover, our system is designed to greatly simplify the burden 
imposed on the user end; only a single image from a sequence which includes a nematode in its environ- 
ment must be provided. From this input, models for both the background and the nematode appearance 
are individually learned using MOGs (see Methods). These models are then applied to segment the ne- 
matode in subsequent images. Next, we provide an original algorithm for extracting nematode skeletons 
for applications to C. elegans^ behavioral assays. Nematode segmentation and skeletonizing algorithms 
have been packaged together in a software for straightforward use by a broad range of researchers. We 
test our image analysis system on representative locomotive environments (Fig. [I]) and compare perfor- 
mances with a state-of-the-art thresholding method (see Results). Finally, we illustrate some examples 
of motility metrics {e.g. body curvature) which are commonly sought from nematode skeleton data (see 
Discussion). We discuss future directions for algorithmic improvement {e.g. multi-worm tracking) as well 
as new directions for potential applications {e.g. cell tracking). 

Methods 

The Multi-Environment Model Estimation (MEME) framework consists of two sequential stages. As a 
first step, (i) the user provides information, allowing a model for the nematode and the background 
environment to be learned. In the second stage, (ii) the nematode and background models arc used to 
segment the nematode and then extract its skeleton for a sequence of images. In stage (i), the user is 
required to input a hand-segmentation of the nematode and its corresponding width for a single image (see 
Supplementary Video SI). This approach can be viewed as a form of "One Shot Learning" [l3{[T5] , where 
model learning occurs only once, from a single labeled example. A flowchart of the MEME framework is 
schematically shown in Fig. [3j 

We briefly introduce the notation used throughout the article. We define the sequence of images pro- 
vided by the user as I = {/i, . . . ,/7v} for A^ discrete time steps, whereat each time step t. It £ [0,255]"^™; 
n and m are the width and height of each image, respectively. We denote the user provided data {U) as 
a triple U = {Ijj-, Su,Wu}, where Ijj is an image from I, Sjj is the nematode body segmentation and 
Wu is the nematode width (Fig. [s]). We specify the intensity models derived from U as Fw for the worm 
and Fb for the background. For any given image It included in I, we define the computed segmentation 
of the nematode as St and the list of ordered pixel coordinates describing the nematode skeleton as L*. 

Nematode Segmentation 

Our first step is to provide an automatic mechanism to compute accurate nematode segmentations for 



a single target using a static camera (as in 41 52]). To do this, we build on the idea of using Mixtures 
of Gaussians (MOG) [34|[44 50 to construct accurate and robust appearance models for the background 
and the nematode. As it is often the case for MOG methods, each pixel in an image is treated as a 
random variable which can be modeled by summing K weighted Gaussian distributions. This can be 
formally written as 

K 

P(x)=^^,5(j|Ai„S,), (1) 

where x is a pixel intensity value, tt^,^, and Sj are respectively the weight, mean and covariance of 
the ith Gaussian distribution Q. These parameters are usually estimated by means of an Expectation- 



Maximization (EM) algorithm 29 . The intuition behind this model is that each individual Gaussian 
represents the appearance a particular pixel may take. Therefore, combining each Gaussian provides a 
way to model complex pixel observations. Typically, doing this over all pixels in an image is an effective 



way to model background scenes 34 44 50 . 



A consequence of this approach is that the number of MOGs used is considerable {i.e. the total 
number nm of pixels in an image) and the number of parameters required is very large {3Knm). In 
turn, a substantial number of images is needed to estimate the MOG parameters accurately as each 



image only provides a single sample for each MOG (see 44 for more details). Moreover, the entire 
background scene must be visible when attempting to estimate these parameters, since each image is 
used to model the background and not the nematode. This latter condition becomes problematic when 
image sequences always contain a nematode in the field of view {e.g. Supplementary Videos available in 



references (3lp2 35 ) 



To avoid such drawbacks, we choose instead to model the nematode appearance (-FV) in addition to 
the background model {Fb) by means of MOGs. To learn the parameters of Fw, we use the information 
gathered from the user. Namely, Su provides the pixel region of lu containing the nematode as delineated 
by the user (Fig. Ilk). From this region, we randomly select pixel locations and extract [d x d] image 
patches from I[j around each location. These patches are then vectorized and treated as independent 
samples. We denote this feature extract process as f{u, u; /, d) = x € M'^ , where we select a patch 
around pixels (u, v) for any given image /. Notice that when d — 1, this reduces to sampling the selected 
pixels only; Fig. |4Fb) shows the histogram of intensities for the case d = 1. In general, applying this 
transformation allows for modeling intensities with respect to image patches, as this approach carries 
more information than individual and independent pixels. Computing an appropriate value for d is done 
by using a linear model {i.e. d = ai-_ — ^K^ — y+ctoj where ai and ao are constants). The samples extracted 
are then used to estimate the parameters of Fw by using the EM algorithm. Figure W[d) illustrates a 
visual representation of the estimated MOGs of Fw for K = 2 and d — 1. 

Next, modeling of the background {Fb) is done by partitioning the image {lu) into fi distinct and 
non-overlapping cells, C = {Ci, . . . , C^}, as shown in Fig.lllc). Each cell {Cc) is then treated as a random 
variable and modeled with its own independent MOG. Hence, Fb — {Fg , . . . ,Fg}, where each pixel 
in It is associated with a unique Fg"; in our implementation, we choose n = 10 x 10. The parameters of 
each Fg" are computed from extracted samples in the partition Cc. Similarly to building the nematode 
model, samples are [d x d] pixel patches from lu, which have been vectorized. Examples of intensity 
distributions for two arbitrary cells (Fig. l4J:) are displayed in Fig. [4|b) along with their corresponding 
MOG representations in Fig. Hd). Two consequences arise from such partitioning. First, only a total 
of 3Kh parameters need to be estimated, as opposed to 3Knm. Secondly, a single image is sufhcient 
to estimate these parameters, as background regions covered by the nematode can still be modeled by 
neighboring pixels in a cell. This reduces the number of training images required and relaxes the constraint 
that training images must only contain background pixels. Note that when a cell is reduced to a single 



pixel {h sa nm), Fb is similar to the model described in 41 52 . Alternatively, when a cell corresponds 
to the entire image {n — 1), Fb is similar to models typically used by thresholding techniques fl 21 22 



as 



Nematode segmentation for image It can then be computed at each pixel {u,v), belonging to cell Cc, 

St{u,v)^{ I ^[r{f{u,v;It,d),c)>l, 
^ ' ^ I otherwise, ^ ' 



where 



ri^^-) = ^^^-y /'k:r'rh:Zc' - (3) 



Fw{x) _f^ 7rrg{x\nr ^^-^ 



The procedure above allows one to compute the nematode segmentation for a given image It, where 
K — 2 in our system. Notice that using the ratio of MOGs (see Eq. pi) is an effective way to avoid 
any form of thresholding. This is due to the fact that both Fw and Fb are explicitly modeled. Finally, 
opening and closing morphological operations are used to smooth nematode segmentations. 



Nematode Skeleton 

Over the years, a large number of methods have been used to extract skeletons from segmented nema- 



todes. Methods have ranged from using specific nematode models 22 41 , to heuristically constructing 
the nematode's medial axis flll20i 21, 23, 41j . While these various methods have shown success, they are 
generally influenced by the quality of the segmentation. In an attempt to reduce sensitivity to segmenta- 
tion noise, we propose an original algorithm which balances geometric features (i.e. nematode boundary) 
and global shape {i.e. nematode undulating posture) in a seamless framework. The proposed method 
has the advantage of being intuitive and simple to implement. 

We cast our problem once again in a probabilistic manner such that the nematode skeleton (Lt) is 
considered to be a sequence of discrete unknown skeleton pixel locations {Lt = {li, . . . , Im}), where each 
location is a point on the skeleton and must be determined. It is assumed here that either the head or tail 
pixel location (/q) is initially known; determining Lf is then viewed as a sequential Bayesian estimation 
problem 124,26,49 . Given the initial position ^o, we infer the location of the next point (^i) by observing 
the likelihood of potential locations {e.g. the likelihood of a pixel being ^i) as well as the history of 
directions between subsequent pairs of points {e.g. from Iq to h). The 'skeleton' algorithm is iterative 
such that a new location along Lt is inferred at each iteration step (t). To infer all points in Lt, this 
process is simply repeated. 

First, the input of our algorithm is the segmentation of the nematode for a given image {St). A 
skeleton pixel location is defined as Ir = {uttVt), where u-r and v-r are pixel locations in St- Let v be 
a discrete random vector describing the direction from Ir to Ir+i, such that u E V = { — 1,0,1}^. This 
corresponds to It being one pixel away from Ir+i- Let Pt{i^) be the corresponding probability distribution 
of v at iteration step r. As more skeleton pixels are inferred, the distribution Pt{i^) will evolve. Initially 
this distribution is uniform, as no prior information between l^. and Ir+i is known. The initial position 
{Iq) is found by using maximal response locations when running a coarse corner detector on St- Selecting 
the following point on the skeleton can then be computed by maximizing the Maximum a Posteriori 
(MAP) estimator, 

Z^_l_2^ = It + argTaaxP{lT\i')PT{i^), (4) 

where P{It\v) is the likelihood that direction v leads to the next skeleton point and is modeled by 

Here, Dil) is the distance computed when applying the Chamfer distance transform (2 16 to St- This 
transformation computes the Euclidean distance of each pixel in St to its closest nematode boundary 
pixel. An example of this distance transform is shown in the contour plot of Fig. [5] Here, the boundary 
of the nematode has a distance of zero, while values of D increase steadily for pixels approaching the 
medial axis of the nematode. Equation Q then implies that skeleton locations are picked by (i) weighing 
how likely pixels are to be at the center of the segmented nematode, combined with (ii) the history of 
the chosen vector directions. This strategy is particularly useful in cases where the segmentation is noisy, 
as the history of vector directions guides where the following pixel location should be located. In order 
to remove the possibility of selecting the same pixel several times, l-r is removed from possible future 
locations by setting D{It) — 0. 

Once Ir+i is determined, the distribution Pt{i^) must be updated for the following iteration. Using 
Bayes rules, Pr+i{h') is computed for Vi^ G V by 

Pr+l{iy) = ^P{lrW)Pr{iy): (6) 

where Z is a normalization constant. 



Inferring Lt = {li, . . . , Im} for a given image It then consists in the following algorithm. First, ?o is 
given at iteration step r = 0. Three steps are then repeated: (i) compute the next skeleton location from 
Eq. Q; (ii) update the distribution of v from Eq. ([6]); and finally (iii) increment the iteration step. These 
operations are repeated until a point on the boundary is encountered (i.e. D{Im) ~ 0). An example of 
the resulting Lt (skeleton) is shown in the inset of Fig. [5] 

Results 

We aim at providing a versatile nematode segmentation framework with performances comparable to, 
or better than, state-of-the-art image analysis systems [l| [II|[T4t[l7l[20}|23l[53) . To compare the MEME 
framework against such systems, we evaluate our algorithm quantitatively for a series of image sequences 
obtained for various C. elegans locomotive environments. The sequences include one or more data sets 
for behavioral assays such as (i) crawling on substrates (Supplementary Video S2 and S3), (ii) swimming 
in a drop (Supplementary Video S4), (iii) swimming in shallow acrylic channels (Supplementary Video 
S5), (iv) locomotion in a gelatin-based solution (Supplementary Video S6), and (v) locomotion in a 
microfluidic substrate (Supplementary Video S7). A total of 13 image sequences are investigated (see 
Supplementary Table SI for complete listing and data source). In each sequence, the target nematode 
is present in all images. The MEME framework is implemented using Matlab; computing the nematode 
segmentation and skeleton for a 640 x 480 pixel size image requires approximately 1 second on a standard 
PC {i.e. 2.0 Ghz). 

The state-of-the-art method of choice for comparison with MEME is an in-house developed thresh- 



olding algorithm W7 48 , similar to standard intensity-based threshold approaches 11 14l ITlpOlpS] . To 
perform a fair comparison between MEME and the thresholding framework, both methods are initially 
provided with a single image to tune their respective parameters. As described for MEME (see Methods), 
the user selects from the initial image (i) the nematode region and (ii) the nematode width (Supplemen- 
tary Video SI). For the threshold-based method, all images of a sequence are first used to compute a 
background image of the environment by pixel averaging. Background subtraction is then applied to each 
image. Next, several thresholds are used to prune the remaining background pixels. These are manu- 
ally selected by optimizing segmentation results on the initial image (Supplementary Table SI). Finally, 
opening and closing morphological operators are used to smooth and discard final background regions. 
Note that in the case where the number of images in the sequence is small, background subtraction is 
omitted and only threshold intensities are used. 

In order to quantitatively evaluate any segmentation algorithm, results must be compared to a ground 
truth [50| . For the present purpose, the ground truth is set as the true, or optimal, nematode segmentation 
provided by an expert. Hence, we manually segment a small set of images (n = 30 - 40) from each 
sequence {e.g. Fig.pj second row) and compare the performance of each algorithm to this image sub-set. 
Determining such ground truth allows for a precise definition of correct and incorrect pixel classification. 



The performance of a segmentation algorithm can be evaluated by measuring two distinct metrics 50 
(i) the surface error and (ii) the nematode yield. The former quantity computes the proportion of pixels 
which are misclassified by the algorithm over the entire image. This metric is mathematically defined as 

^' = ^ E \Gt{p)-St{p)\, (7) 

where p is a pixel location and Gt is the ground truth image for It. Hence, Ct attributes equivalent weight 
to errors on the background and the nematode regions. The nematode yield, however, only computes 
the proportion of the nematode region which is correctly segmented. Consequently, misclassified pixels 
belonging to the background have no impact on the nematode yield. This metric is defined as 



^* = M ^ \Gtip) - Stip)l (8) 

where Af is the set of pixels which satisfy Gt{p) — 1. Together, et and rjt provide a quantitative and 
rehable measure of segmentation performance 50 1 . 

Quahtative segmentation results are shown for a selection of motility environments in Fig.lGlas well as 
in the Supplementary Videos S2 to S7. In general, the MEME method is capable of segmenting nematodes 
at least as well as the thresholding method. For the "Crawl" , "Drop" , and "Channel" environments (first 
three columns. Fig. pi), both methods yield qualitatively similar results. Here, the environments illustrate 
a relatively homogenous background. However, in the complex "Microfluidic" environments (Fig. pi), 
results contrast more sharply between the two approaches; MEME provides cleaner segmentations which 
capture more closely the original shape of the nematodes. 

Figure ff\ shows the results from the computation of the surface error (Fig. [7k) and the nematode 
yield (Fig.lTp) for the environments of Fig. K\ Data for et and rjt is reported in the Supplementary 
Table SI for the complete 13 image sequences. In general, computations of surface error (e^) illustrate 
comparable performances between MEME and the threshold-based approach (Fig. ITk). Yet, in two 
complex "Microfluidic" environments, MEME does significantly better. Note that for all environments 
investigated here (Fig. [7k and Supplementary Table SI), et remains below 10%. In fact, for homogeneous 
background environments such as "Crawl", "Drop", and "Channel", et ^ 1%, emphasizing the good 
results obtained both by MEME and the threshold-based approach. 

We observe, in contrast, significant improvements in nematode yield (rjt) when using MEME compared 
to the thresholding method (Fig. [tId). From the set of 13 assays tested here, 10 cases show examples of 
MEME significantly outperforming the threshold-based method (Supplementary Table SI); in some cases, 
with margins greater than 20 percentage points {e.g. "Microfluidic" and "Microfluidic H", Fig. [tId). In 
the remaining environments where the thresholding method performs relatively better {e.g. "Microfluidic 
III", Fig. [rb), the differences in rjt remain however small, i.e. between 1.79 and 4.71 percentage points. 
Overall, our MEME algorithm outperforms the threshold-based approach for the overwhelming majority 
of cases examined. 

Discussion 

Our experiments show that MEME provides a reliable framework to obtain nematode segmentations of C. 
elegans across various locomotive environments. In addition, MEME offers signiflcant improvements over 
alternative image analysis systems available; these include (i) better, or similar, performances compared 
to state-of-the-art thresholding approaches i47, 48 , (ii) no ncmatode-free image sequence required for 



learning background appearances 41 52 ; and (iii) a small amount of user input needed, i.e. a single 
hand-segmentation of the nematode and a marking of its width (Supplementary Video SI). This last 
improvement is particularly attractive from a user point of view as substantial effort may be needed with 
thresholding techniques to obtain similar results. Overall, these attributes make the MEME framework 
both attractive and straightforward to use for a broad range of researchers. 

While computing good nematode segmentations with threshold-based methods is possible (Fig.l7|, this 
process can quickly become laborious. Indeed, several iterations are required by the user to find optimal 
thresholds for a given environment (Supplementary Table SI). The main complication arises from the non- 
uniform backgrounds and appearance {i.e. pixel intensities) which characterize many environments. For 
example, a single threshold is incapable of distinguishing between the nematode and the background in the 
presence of pillars in microfiuidic substrates {e.g. Fig.pj last column). Similarly, single thresholds cannot 
adapt to specific locations in an image. This becomes crucial for accurate segmentation of nematodes in 
environments where lighting conditions may not be uniform {e.g. Supplementary Video S5). 



In general, the improvement observed with MEME can be attributed to two main reasons: (1) the 
nematode appearance model is explicitly learned and used to help decide whether pixels belong to the 
nematode. In practice, when using threshold-based methods, many of the regions which are considered 
"not background" after applying a threshold do not resemble the nematode at all {e.g. pillars in "artificial 
dirt" assays of Lockery et al. fS?", Fig. [TV and Supplementary Video S7). Using both the nematode and 
background appearance models significantly reduces the need of using intense pruning schemes to reject 
such regions. (2) The background scene is partitioned into a grid of sub-regions (Fig. l4b), where each cell 
is explicitly modeled. This allows for local variations in intensities to be grouped by region, providing 
a localized statistical model for each area of the background scene. This strategy has the advantage of 
appropriately modeling backgrounds where large variations in lighting occur {e.g. Fig. nt). 

In cases where the nematode appearance differs significantly from the background, such as in crawling 
and swimming assays {e.g. Fig. flk and b), we observe nematode yields {rjt) beyond 80% (Fig. m and Sup- 
plementary Table SI). In contrast, more complex environments can substantially reduce this performance 
{e.g. microfluidic substrates). The main difficulty therein lies in that only pixel intensities are modeled; 
this represents an important limitation when pixel intensities of the nematode and the background are 
too similar. A typical illustration of this problem occurs at the head and tail of C. elegans, where the 
nematode extremities are often transparent against the background. For example, this problem is ob- 
served in microfiuidic substrates {e.g. Fig. fife and f) where the ends of the nematode are lost during the 
segmentation process. A direct consequence of this is the truncated length of nematode skeletons {e.g. 
inset of Fig [s] and Supplementary Videos S7). 

Our MEME framework is currently optimized for segmenting a single target nematode within an 
image sequence. Nevertheless, scenarios where multiple nematodes enter the scene in subsequent images 
are still supported by our algorithm as long as only one nematode is present in the input image. That 
is, an arbitrary number of nematodes may be segmented for a given image sequence. Note, however, 
that cases where the appearance of either the nematode or the background changes significantly over 
the length of an image sequence will cause improper segmentations. Furthermore, extracting skeletons 
remains a challenge in some scenarios. For example, cases where the nematode coils on itself, or when 
its head and tail touch {e.g. omega bend), are currently not supported with the implemented skeleton 
algorithm. In the former case, the segmentation simply does not provide a correct shape representation 
of the nematode {i.e. a closed circle as opposed to a 'snake'). The problem lies in the fact that estimating 
the medial axis of the nematode with the Chamfer distance transform is ill-suited. In principle, the latter 
scenario {i.e. omega bend) is not problematic. In practice, however, initializing the skeleton algorithm is 
ill-posed; there is no a priori knowledge as to where the head or tail lie. 

Motility Metrics 

We briefly discuss the feasibility of using nematode skeletons obtained with MEME (Supplementary 
Videos S2 to S7). Skeleton data often provides the building blocks to quantify locomotive traits of C. 



elegan s [27 28l 35 , 47l 48] . Here, we illustrate some of these motility metrics across sample environments. 
In Fig. |8|(top row), nematode tracking data is shown over multiple body bending cycles for crawling on a 
substrate (left column), swimming in a drop (middle column), and locomotion in so-called "artificial dirt" 
(right column), i.e. a microfluidic substrate (Supplemental Videos in Lockery et al. 32 ). Trajectories 
swept by the nematode tail (or head) are labeled, illustrating striking differences in the travel paths 
adopted by C. elegans as a function of the surrounding environment. Snapshots of nematode skeletons 
over one beating cycle are shown in Fig. Is] (middle row); the time evolution of skeletons is color-coded as a 
function of the corresponding beating period (T) . Plots reveal the existence of well-confined envelopes of 
body postures which vary dramatically with motility assay. Here, envelopes of postures are constructed 
using a principal component analysis (PCA) to find the skeleton's principal axis and orientation at each 
instant in time. Further metrics including the nematode wavelength as well as the amplitude of body 



undulations can be obtained in a straightforward manner from the construction of such envelopes 47 . 



Next, we illustrate measures of body curvature (k) along the nematode's length (Fig. [SJ bottom row); 
such plots have been shown to characterize swimming and crawling gaits [28[|35|[47| . Curvature is defined 
as k(s, t) = d(j)/ds, where is the angle made by the tangent to the x-axis at each point along the nematode 
skeleton; s is the arc-length coordinate spanning the nematode's head (s — 0) to its tail (s = L). The 
spatio-temporal evolution of k is shown over several beating cycles for each environment. Here, curvature 
values are color-coded; red and blue represent positive and negative values of k, respectively. Note that 
the vertical axis in each contour plot corresponds to the non-dimensional body position {s/L), where L 
is the nematode length. Each contour plot shows the existence of highly periodic, well-defined diagonally 
oriented lines. These diagonal lines are characteristic of bending waves of motion which propagate in 
time along the nematode body length {i.e. traveling waves). 

In Fig.lsl forward motion displays curvature lines with a positive slope (left and middle column); waves 



are initiated at the nematode head 35 47 . Conversely, backward motion displays lines with a negative 
slope, where bending motion is initiated at the tail (right column). In general, a number of motility 
metrics may be directly extracted from such curvature contour plots. For example, the body bending 
frequency (/) may be obtained by applying a one-dimensional (ID) Fast Fourier Transform (FFT) to 
the curvature field k at multiple body positions s/L [47]. Similarly, the wave speed (c) may be directly 
extracted from the slope of the curvature k propagating along the nematode's body; the wavelength 
A = c/f is then computed in a straightforward manner. With our MEME platform, nematode skeleton 
data is made ready available for use for motility analysis of C. elegans. 

Future Directions 

The proposed MEME framework provides researchers with an attractive and reliable platform for nema- 
tode segmentation and 'skeletonizing' across a large spectrum of C. elegans motility assays. The MEME 
software is freely available upon request (contact person: J. Sznitman; website for download will be 
provided) . Improving our system to further assist researchers conduct quantitative analysis of C. elegans 
is of course desired. In the near future, one immediate goal is to provide segmentations and skeletons 
simultaneously for multiple nematodes. This 'upgrade' would be of great interest for high-throughput 



drug screening applications 40 42] . From a performance point of view, combining larger sets of image 
features {e.g. edges, texture, etc.) with MOG models may provide better appearance models for difficult 
environments. This may yield better segmentation results, in particular at the nematode extremities (i.e. 
head and tail). Finally, our MEME platform is not restricted to image analysis of C. elegans only. For 



instance, MOG methods may be used for applications relating cell tracking and motility 12 30,55 1. We 
illustrate here an example of such possible application with Albino Swiss mouse embryo fibroblast cells 
(Supplementary Video S8). 

Supporting Information 

Table SI: Compiled data on segmentation results for the Multi-Environment Model Estimation (MEME) 
and threshold-based algorithms. Performances of each algorithm {i.e. surface error and nematode yield) 
are evaluated for 13 different image sequences representative of various locomotive environments {e.g. 
crawling on agar plate, swimming in a channel or a drop, locomotion in microfluidic substrates). 

Video SI: MEME software tutorial shown for a sample image sequence (source: berrigel2.0perc.mov. 
Supplementary Material in Berri et al. [S]). 

Video S2: Example of crawling assay. From left to right: (i) raw image, (ii) binary segmentation from 
MEME, and (iii) resulting skeleton superimposed on raw image. Here, a young adult, wild-type (N2) 
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C. elegans is seen crawling on an agar plate. Nematode is approximately 1 mm long (image resolution: 
1/78 mm/pix; image acquisition rate: 28 frames per second). 

Video S3: Example of crawling on a substrate. From left to right: (i) raw image, (ii) binary segmentation 
from MEME, and (iii) resulting skeleton superimposed on raw image. The original data is obtained from 
the Supplementary Information (Movie 1) in Pierce-Shimomura et al. [35| . 

Video S4: Example of swimming assay in a liquid drop. From left to right: (i) raw image, (ii) binary 
segmentation from MEME, and (iii) resulting skeleton superimposed on raw image. Here, a young 
adult, wild- type (N2) C. elegans is seen swimming in a 5 ml drop of M9 buffer solution. Nematode is 
approximately 1 mm long (image resolution: 1/78 mm/pix; image acquisition rate: 28 frames per second). 

Video S5: Example of swimming assay in a shallow channel. From left to right: (i) raw image, (ii) binary 
segmentation from MEME, and (iii) resulting skeleton superimposed on raw image. Here, a young adult, 
wild-type (N2) C. elegans is seen swimming in a narrow acrylic channel filled with M9 buffer solution. 
Details are given in Sznitman et al. |47;. Nematode is approximately 1 mm long (image resolution: 
1/400 mm/pix; image acquisition rate: 125 frames per second). 

Video S6: Example of nematode locomotion in a gelatin-based solution. From left to right: (i) raw 
image, (ii) binary segmentation from MEME, and (iii) resulting skeleton superimposed on raw image. 
The original data is obtained from the Supplementary Information (berrigel2.Gpcrc.inov) in Bcrri et al. [3]. 

Video S7: Example of nematode locomotion in "artificial dirt", i.e. a microfluidic substrate. From left 
to right: (i) raw image, (ii) binary segmentation from MEME, and (iii) resulting skeleton superimposed 
on raw image. The original data is obtained from the Supplementary Information (Video 2) in Lockery 



et al. 32 



Video S8: Application of the MEME software to segmentation of Albino Swiss Mouse Embryo Fibroblast 
cells (3T3 Line) . The original video (Video 2) is extracted from live-cell imaging videos available at Nikon 
Microscopy U (http://www.microscopyu.com/moviegallery/livecellimaging/3t3/index.html), as obtained 
with Differential Interference Contrast (DIG) microscopy. 
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Figure Legends 




Figure 1. Examples of environments used in C. elegans'' motility assays, (a) Nematode 
crawling on an agar plate (Supplementary Video S2). (b) Nematode swimming in a 5 ml drop of M9 
buffer solution (Supplementary Video S4). (c) Nematode swimming in a solution of gelatin dissolved in 
M9 (source: berrigelO.Operc.mov, Supplementary Material in Berri et al. 13]). (d) Nematode swimming 
inside a fluid-filled chamber (source: SM2.avi, Supplementary Material in Pierce-Shimomura et al. 
(e)-(f) Nematode locomotion in a microfluidic substrate (source: Supplemental Videos 2 and 4 in 
Lockery et al. i32,). (g) Nematode swimming in a shallow acrylic channel filled with M9 
(Supplementary Video S5). Nematodes shown in (a) through (g) are wild-type (N2) C. elegans and are 
all approximately 1 mm long. 
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Figure 2. Pixel intensity distributions of nematode and background environment. Plots (a) 
through (g) correspond to distributions obtained from the images of Fig. fll Grayscale pixel intensities 
vary between (black) and 255 (white). Distributions are assumed to be Gaussian and parametrized 
with the mean and standard deviation of pixel intensities. 
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Figure 3. Multi-Environment Model Estimation (MEME) framework overview. The system 
consists of two components: (1) a user input and learning stage and (2) an image analysis stage. In 
stage (1), the user provides an image (lu) smd marks the nematode boundary (Su) and width (Wu). 
From this input, appearance models for the nematode {Fyy) and background (Fb) are learned. In (2), 
for each image (It) in a sequence, nematodes are then segmented (St) by using Fyy and Fb- Nematode 
skeletons (Lt) are then extracted from these segmentations. 
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Figure 4. User input and nematode segmentation. Figures (a) through (d) illustrate the stages 
of the segmentation process in MEME for a sample environment (source: Supplemental Video in 
Lockcry et al. [32 ). (a) The user selects the boundary of the nematode on a given image. From this 
manual segmentation, the distribution of nematode features can be extracted, (b) Distribution of pixel 
intensities from the nematode region. Here, d = 1 for illustrative purposes, (c) The background scene is 
partitioned into a grid, where each cell corresponds to a particular pixel block. Two arbitrary cells are 
labeled for clarity; their corresponding intensity distributions are shown in (b). For both the nematode 
and the cells, MOG parameters are then learned, (d) Representation of the MOG models for the 
nematode {F'^) and the two background cells (-F^^ and F^^). 
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Figure 5. Computing the nematode skeleton. Representation of the Cliamfer distance transform 
field {D) applied to the segmented nematode of Fig. |4J The value associated at each pixel of the image 
is the Euclidean distance (in pix) to the closest point of the nematode boundary; the distance on the 
boundary is zero and higher distances lie towards the nematode medial axis. (Inset) Resulting skeleton 
is achieved by balancing geometric features {i.e. Chamfer distance) and global shape [i.e. nematode 
curvature). 
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Figure 6. Nematode segmentation for various locomotive environments, (top row) 

Snapshots of raw images are respectively shown for crawling (Supplementary Video S2), swimming in a 
drop (Supplementary Video S4), swimming in a channel (Supplementary Video S5), and locomotion in 
various microfluidic substrates (source: Supplemental Videos in Lockery et al. [32]). Comparisons 
between nematode segmentations are respectively shown for (i) the ground truth, i.e. hand-segmented 
nematodes (second row), (ii) a threshold-based approach [47", 48 (third row), and (iii) the 
Multi-Environment Model Estimation (MEME) algorithm (bottom row). See Supplementary Table 
SI for data on all 13 cases investigated. 
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Figure 7. Performance evaluation of nematode segmentation algorithms. Here, the 
Multi-Environment Model Estimation (MEME) algorithm is compared to a state-of-the-art thresholding 
approach 47 48 for the environments shown in Fig. p] (a) Surface error (et): proportion of pixels 
which are misclassified by an algorithm over the entire image (see Eq. (It])). (b) Nematode yield (rjt)'- 
proportion of the nematode region which is correctly segmented (see Eq. (|8|). Complete data on surface 
error and nematode yield is available in the Supplementary Table SI for all 13 cases investigated. 
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Figure 8. Examples of nematode locomotive features in sample environments. Here, 

nematode skeleton data is shown for a crawling assay (left column), for swimming in a 5 ml drop 
(middle column), and for locomotion in a microfluidic substrate obtained from Lockery et al. [321 (right 
column). Additional skeleton data is shown in Supplementary Videos S2 to S7. (top row) Tracking 
data of path swept by nematode head (or tail) over multiple beating cycles, (middle row) Color-coded 
temporal evolution of C. elegans skeletons over one beating period (T). Results reveal a distinct 
envelope of body postures for each environment, (bottom row) Spatio-temporal contour plot of body 
curvature (k) along the length of the nematode's skeleton. Red and blue colors represent positive and 
negative curvature values, respectively. The y-axis corresponds to the dimensionless position (s/L) 
along the C. elegans' body length where s = is the head and s = L is the tail. 



